Scanned beam display with adjustable accommodation

ABSTRACT

A scanning beam display controls the curvature of scanning light wave impinging on the eye to simulate image points of differing depth. To simulate an object at a far distance the generated light waves are flatter. To simulate closer objects, the light wave curvature increases. When changing the curvature of the light waves, the eye responds by altering its focus. The curvature of the light waves thus determines the apparent focal distance from the eye to the virtual object. To vary the curvature, either a variable focus lens or a variable index of refraction device is used. Alternatively, a moving point source is used. The generated apparent distance of a virtual object is correlated to a detected distance in a background field of view. Intensity of the virtual object is correlated to detected intensity of background light.

CROSS REFERENCE TO RELATED APPLICATIONS

This is a divisional of U.S. patent application Ser. No. 09/188,993filed Nov. 9, 1998 of Michael Tidwell et al. for “Scanned Beam Displaywith Adjustable Acommodation.” The content of such application isincorporated herein by reference and made a part hereof.

This invention is related to U.S. patent application Ser. No. 09/009,759filed Jan. 20, 1998 of Charles D. Melville for Augmented Imaging Using ASilhouette To Improve Contrast. This invention also is related to U.S.patent application Ser. No. 09/188,991 filed Nov. 9, 1998 of Charles D.Melville et al. for Method and Apparatus for Scanning Optical Distance.The content of all such applications are incorporated herein byreference and made a part hereof.

BACKGROUND OF THE INVENTION

This invention relates to scanning beam display devices, and moreparticularly to optical configurations for scanning beam displaydevices.

A scanning beam display device is an optical device for generating animage that can be perceived by a viewer's eye. Light is emitted from alight source, collimated through a lens, then passed through a scanningdevice. The scanning device defines a scanning pattern for the light.The scanned light converges to focus points of an intermediate imageplane. As the scanning occurs, the focus point moves along the imageplane (e.g., in a raster scanning pattern). The light then divergesbeyond the plane. An eyepiece is positioned along the light path beyondthe intermediate image plane at some desired focal length. An “exitpupil” occurs shortly beyond the eyepiece in an area where a viewer'seye is to be positioned.

A viewer looks into the eyepiece to view an image. The eyepiece receiveslight that is being deflected along a raster pattern. Light thusimpinges on the viewer's eye pupil at differing angles at differenttimes during the scanning cycle. This range of angles determines thesize of the field of view perceived by the viewer. Modulation of thelight during the scanning cycle determines the content of the image.

For a see-through display, a user sees the real world environment aroundthe user, plus the added image of the scanning beam display deviceprojected onto the retina. When the user looks at an object in the fieldof view, the eye performs three basic functions. For one function, eacheye moves so that the object appears at the center of vision. For asecond function, each eye adjusts for the amount of light coming intothe eye by changing the diameter of the iris opening. For a thirdfunction, each eye focuses by changing the curvature of the eye lens. Ifthe focal distance from the third function does not match the distanceto the point of convergence, then the brain detects a conflict. Nauseamay occur.

SUMMARY OF THE INVENTION

According to the invention, a more lifelike image is generated with avirtual retinal display by including a method and apparatus of variableaccommodation.

According to one aspect of the invention, the scanning beam displaydevice controls the curvature of scanning light waves impinging on theeye to simulate image points of differing depth. Images at far distancesout to infinity have flat light waves impinging the eye. Images at neardistances have convex-shaped light waves impinging the eye. Thus, tosimulate an object at a far distance the light waves transmitted fromthe display to the eye are flat. To simulate closer objects, the lightwave curvature increases. The eye responds to the changing curvature ofthe light waves by altering its focus. The curvature of the generatedlight waves relates to a desired, ‘apparent distance’ between a virtualobject and the eye.

According to another aspect of the invention, a variable focus lens isincluded in the virtual retinal display to alter the shape of the lightwaves. The lens varies its focal length over time as desired. Forexample, for an image that is 640 by 480 pixels, there are 307,200 imageelements. The variable focus lens is able to adjust its focal lengthfast enough to define a different focal length for each image element.

According to another aspect of the invention the variable focus lens isformed by a resonant crystalline quartz lens. The resonant lens changesthickness along its optical axis, thus varying its focal length. Thelens varies in focal length with respect to time. By varying the timewhen a light pulse enters the resonant lens, the focus is varied. Anon-resonant lens is used in another embodiment where its response timeis fast enough to focus for each image element.

According to another aspect of the invention, a device which changes itsindex of refraction over time is used instead of a variable focus lens.In one embodiment an acousto-optical device (AOD) or an electro-opticaldevice (EOD) is used. In the AOD, acoustic energy is launched into anacousto-optic material to control the index of refraction of the AOD. Inone embodiment of an EOD, a lens is coated with a lithium niobate layer.An electric field is applied across the lithium niobate material to varythe index of refraction of the coating. Changing the index of refractionchanges the effective focal length of the lens to vary the focusdistance of the virtual image.

In another embodiment an optical device changes its index of refractionbased upon the intensity (frequency) of an impinging infrared beam. Thecurrent intensity of the infrared beam in effect sets the current indexof refraction for the device. Varying the intensity of the infrared beamvaries the index of refraction to vary the effective focal length of theoptical device.

Another embodiment includes a compressible, cylindrical gradient indexlens as a focusing element. A cylindrical piezoelectric transducercompresses an outer shell of the gradient index cylinder. Compression ofthe cylinder shifts the physical location of the lens material tochanges the index of refraction gradient, thereby changing the focallength. Another embodiment includes a current driven device that usesfree-carrier injection or depletion to change its index of refraction.

According to another aspect of the invention, a variable focus lensserves to correct the curvature of the intermediate image plane forerrors introduced by the scanners or from the aberration of otheroptical elements. In an exemplary embodiment, a aberration map of thesystem is stored in a look-up table in memory. The aberration mapprovides correction data for each image element. The correction datadrives the variable focus element to adjust the focal depth for eachimage element.

According to another aspect of the invention, the light source is movedto vary the focal length instead of introducing a variable focus lens tovary the focal length.

According to another aspect of the invention, the light source emitslight toward a mirror that reflects the light toward a lens of thedisplay. The mirror is movable about an axis causing the angle ofreflection to vary. A control signal determines the position of themirror and thus the angle of reflection. As the angle of reflectionvaries, the focal distance of light exiting the lens variesproportionately.

According to another aspect of the invention, an augmented displayincludes variable accommodation. The scanning beam display is augmentedto include a background image upon which a virtual image is augmented.An object within the virtual image is scanned to have an apparentdistance within the field of view. Thus, a virtual object may be placedwithin a real world background view. The apparent distance is controlledby controlling the curvature of the light waves which scan the objectpixels onto the viewer's eye.

According to another aspect of the invention, distance of a backgroundimage object is measured and used to specify the apparent distance of avirtual object to be placed in proximity to such background imageobject.

According to another aspect of this invention, the intensity of avirtual image is controlled relative to measured intensity of abackground image. As a result, the relative contrast between the virtualimage and background image may be the same even within differentbackground image intensities. Further, the virtual image intensity canbe controlled to be approximately the same as the background image for amore realistic viewing effect.

One advantage of varying the curvature of light is that the producedimage is more life-like, enhancing the user's feeling of presence. Theseand other aspects and advantages of the invention will be betterunderstood by reference to the following detailed description taken inconjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a virtual retinal display according to anembodiment of this invention;

FIG. 2 is an optical schematic of the virtual retinal display accordingto an embodiment of this invention;

FIG. 3 is an optical schematic of the virtual retinal display accordingto another embodiment of this invention;

FIG. 4 is an optical schematic of a virtual retinal display without avariable focus lens;

FIG. 5 is an optical schematic of the virtual retinal display accordingto another embodiment of this invention;

FIG. 6 is an optical schematic of the virtual retinal display accordingto another embodiment of this invention;

FIG. 7 is an optical schematic of another virtual retinal displaywithout a variable focus lens;

FIG. 8 is a diagram of light directed toward an eye for depicting lightcurvature for sequential image elements;

FIG. 9 is a perspective drawing of an exemplary scanning subsystem forthe display of FIG. 1;

FIG. 10 is a diagram of a variably transmissive eyepiece for anembodiment of the display of FIG. 1;

FIG. 11 is a diagram of an electro-mechanically variable focus lens foran optics subsystem of FIG. 1 according to an embodiment of thisinvention;

FIG. 12 is a diagram of an alternative variable focus lens embodimentfor the optics subsystem of FIG. 1;

FIG. 13 is a diagram of another alternative variable focus lensembodiment for the optics subsystem of FIG. 1;

FIG. 14 is a diagram of a plurality of cascaded lens for the opticssystem of FIG. 1 according to an embodiment of this invention;

FIG. 15 is an optical schematic of a virtual retinal display accordingto another embodiment of this invention;

FIG. 16 is an optical schematic of a virtual retinal display accordingto another embodiment of this invention;

FIG. 17 is a diagram of an optical source with position controller ofFIGS. 10 and 11 according to an embodiment of this invention;

FIG. 18 is a diagram of an optical source with position controller ofFIGS. 10 and 11 according to another embodiment of this invention;

FIG. 19 is an optical schematic of a virtual retinal display accordingto another embodiment of this invention;

FIG. 20 is a diagram of a display apparatus embodiment of this inventionmounted to eyeglasses that serve as an eyepiece for the displayapparatus;

FIG. 21 is a diagram of a scanning beam augmented display embodiment ofthis invention; and

FIG. 22 is a diagram of a control portion of the display of FIG. 21.

DESCRIPTION OF SPECIFIC EMBODIMENTS Overview

FIG. 1 is a block diagram of a scanning light beam display 10 havingvariable accommodation according to an embodiment of this invention. Thedisplay 10 generates and manipulates light to create color or monochromeimages having narrow to panoramic fields of view and low to highresolutions. Light modulated with video information is scanned directlyonto the retina of a viewer's eye E to produce the perception of anerect virtual image. The display 10 is small in size and suitable forhand-held operation or for mounting on the viewer's head. The display 10includes an image data interface 11 that receives a video or other imagesignal, such as an RGB signal, NTSC signal, VGA signal or otherformatted color or monochrome video or image data signal. Such signal isreceived from a computer device, video device or other digital or analogimage data source. The image data interface generates signals forcontrolling a light source 12. The generated light is altered accordingto image data to generate image elements (e.g., image pixels) which forman image scanned onto the retina of a viewer's eye E.

The light source 12 includes one or more point sources of light. In oneembodiment red, green, and blue light sources are included. The lightsources or their output beams are modulated according to the input imagedata signal content to produce light which is input to an opticssubsystem 14. Preferably the emitted light is spatially coherent.

The scanning display 10 also includes an optics subsystem 14, a scanningsubsystem 16, and an eyepiece 20. Emitted light passes through theoptics subsystem 14 and is deflected by the scanning subsystem 16.Typically light is deflected along a raster pattern, although in analternative embodiment another display format such as vector imaging canbe used. In one embodiment the scanning subsystem 16 receives ahorizontal deflection signal and a vertical deflection signal derivedfrom the image data interface 11. In another embodiment, the scanningsubsystem 16 includes a mechanical resonator for deflecting passinglight.

According to an aspect of this invention the optics subsystem 14includes a device for varying the curvature of light impinging upon theeye E. According to an alternative aspect of the invention, the display10 instead includes a device for moving the light source position withtime to vary the curvature of light impinging upon the eye E.

Embodiments in Which Optics Subsystem Varies Curvature

FIGS. 2-5 show optical schematics for alternative embodiments in whichthe optics subsystem 14 includes a variable focus lens 22 for varyingthe curvature of light impinging upon the eye E. FIGS. 2 and 3 aresimilar but have the variable focus lens 22 for varying curvaturelocated at different locations. In the FIG. 2 embodiment light frompoint source(s) 12 passes through the variable focus lens 22 thenthrough a collimating lens 24 before travelling to the scanningsubsystem 16 and eyepiece 20. In the FIG. 3 embodiment light from thepoint source(s) 12 passes through a collimating lens 24 then through thevariable focus lens 22 before travelling to the scanning subsystem 16and eyepiece 20. The light passing from the eyepiece 20 to the eye E hasits curvature varied over time based upon the control of variable focuslens 22. For some image elements the curvature is of one contour tocause the eye to focus at a first focal length. For other image elementsthe curvature is of another contour to causes the eye to focus at asecond focal length. By controlling the curvature, the display 10controls the apparent focus of the eye, and thus causes different imageelements to appear to be located at different distances.

FIG. 4 shows an optical schematic of a display without the variablefocus lens 22. Note that the light impinging on the eye E is formed byplanar waves. In such embodiment all optical elements appear at acommon, indeterminate depth.

FIGS. 5 and 6 are similar to FIGS. 2 and 3, but are for an opticssubsystem 14 which converges the light rather than one which collimatesthe light. FIG. 7 shows an optical schematic of a virtual retinaldisplay without the variable focus lens 22. Note that the lightimpinging on the eye E for the FIG. 7 embodiment is formed by planarwaves. In such embodiment all optical elements appear at a commonindeterminate depth. In FIG. 5 light from a point source(s) 12 passesthrough the variable focus lens 22 then through a converging lens 24before travelling to the scanning subsystem 16 and eyepiece 20. In theFIG. 6 embodiment light from the point source(s) 12 passes through aconverging lens 26 then through the variable focus lens 22 beforetravelling to the scanning subsystem 16 and eyepiece 20. The lightpassing from the eyepiece 20 to the eye E has its curvature varied overtime based upon the control of variable focus lens 22.

FIG. 8 shows a pattern of light impinging on the eye. The scanning beamdisplay device controls the curvature of scanning light waves impingingon the eye to simulate image points of differing depth. Images at fardistances out to infinity have flat light waves impinging the eye.Images at near distances have convex-shaped light waves impinging theeye. The light is shown as a sequence of light. For a first imageelement 26 the corresponding light 28 has one curvature. For anotherimage element 30, the corresponding light 32 has another curvature.Light 36, 40, 44 for other image elements 34, 38, 40 also is shown. Asequence of image elements is scanned upon the eye E to generate animage perceived by the eye. To simulate an object at a far distance thelight waves transmitted from the display to the eye are flat. Tosimulate closer objects, the light wave curvature increases. Thecurvature of the generated light waves relates to the desired, ‘apparentdistance’ (i.e., focus distance) between a virtual object and the eye.The eye responds to the changing curvature of the light waves byaltering its focus. The curvature of the light changes over time tocontrol the apparent depth of the image elements being displayed. Thus,varying image depth is perceived for differing portions of the scannedimage.

Light Source

The light source 12 includes a single or multiple light sources. Forgenerating a monochrome image a single monochrome source typically isused. For color imaging, multiple light sources are used. Exemplarylight sources are colored lasers, laser diodes or light emitting diodes(LEDs). Although LEDs typically do not output coherent light, lenses areused in one embodiment to shrink the apparent size of the LED lightsource and achieve flatter wave fronts. In a preferred LED embodiment asingle mode, monofilament optical fiber receives the LED output todefine a point source which outputs light approximating coherent light.

In one embodiment red, green, and blue light sources are included. Inone embodiment the light source 12 is directly modulated. That is, thelight source 12 emits light with an intensity corresponding to imagedata within the image signal received from the image data interface 11.In another embodiment the light source 12 outputs light with asubstantially constant intensity that is modulated by a separatemodulator in response to the image datadrive signal. The light outputalong an optical path thus is modulated according to image data withinthe image signal received from the image data interface 11. Suchmodulation defines image elements or image pixels. Preferably theemitted light 31 is spatially coherent.

Additional detail on these and other light source 12 embodiments arefound in U.S. Pat. No. 5,596,339 to Furness, et al., entitled “VirtualRetinal Display with Fiber Optic Point Source” which is incorporatedherein by reference.

Image Data Interface

As described above, the image data interface 11 receives image data tobe displayed as an image data signal. In various embodiments, the imagedata signal is a video or other image signal, such as an RGB signal,NTSC signal, VGA signal or other formatted color or monochrome video orgraphics signal. An exemplary embodiment of the image data interface 11extracts color component signals and synchronization signals from thereceived image data signal. In an embodiment in which an image datasignal has embedded red, green and blue components, the red signal isextracted and routed to a modulator for modulating a red light pointsource output. Similarly, the green signal is extracted and routed to amodulator for modulating the green light point source output. Also, theblue signal is extracted and routed to a modulator for modulating theblue light point source output.

The image data signal interface 11 also extracts a horizontalsynchronization component and vertical synchronization component fromthe image data signal. In one embodiment, such signals define respectivefrequencies for horizontal scanner and vertical scanner drive signalsrouted to the scanning subsystem 16.

Scanning Subsystem

The scanning subsystem 16 is located after the light sources 12, eitherbefore or after the optics subsystem 14. In one embodiment, the scanningsubsystem 16 includes a resonant scanner 200 for performing horizontalbeam deflection and a galvanometer for performing vertical beamdeflection. The scanner 200 serving as the horizontal scanner receives adrive signal having a frequency defined by the horizontalsynchronization signal extracted at the image data interface 11.Similarly, the galvanometer serving as the vertical scanner receives adrive signal having a frequency defined by the vertical synchronizationsignal VSYNC extracted at the image data interface. Preferably, thehorizontal scanner 200 has a resonant frequency corresponding to thehorizontal scanning frequency.

Referring to FIG. 9, one embodiment of the scanner 200 includes a mirror212 driven by a magnetic circuit so as to oscillate at a high frequencyabout an axis of rotation 214. In this embodiment the only moving partsare the mirror 212 and a spring plate 216. The optical scanner 200 alsoincludes a base plate 217 and a pair of electromagnetic coils 222, 224with a pair of stator posts 218, 220. Stator coils 222 and 224 are woundin opposite directions about the respective stator posts 218 and 220.The electrical coil windings 222 and 224 may be connected in series orin parallel to a drive circuit as discussed below. Mounted on oppositeends of the base plate 217 are first and second magnets 226, the magnets226 being equidistant from the stators 218 and 220. The base 217 isformed with a back stop 232 extending up from each end to formrespective seats for the magnets 226.

The spring plate 216 is formed of spring steel and is a torsional typeof spring having a spring constant determined by its length and width.Respective ends of the spring plate 216 rest on a pole of the respectivemagnets 226. The magnets 226 are oriented such that they have like polesadjacent the spring plate.

The mirror 212 is mounted directly over the stator posts 218 and 220such that the axis of rotation 214 of the mirror is equidistant from thestator posts 218 and 220. The mirror 212 is mounted on or coated on aportion of the spring plate.

Magnetic circuits are formed in the optical scanner 200 so as tooscillate the mirror 212 about the axis of rotation 214 in response toan alternating drive signal. One magnetic circuit extends from the toppole of the magnets 226 to the spring plate end 242, through the springplate 216, across a gap to the stator 218 and through the base 217 backto the magnet 226 through its bottom pole. Another magnetic circuitextends from the top pole of the other magnet 226 to the other springplate end, through the spring plate 216, across a gap to the stator 218and through the base 217 back to the magnet 226 through its bottom pole.Similarly, magnet circuits are set up through the stator 220.

When a periodic drive signal such as a square wave is applied to theoppositely wound coils 222 and 224, magnetic fields are created whichcause the mirror 212 to oscillate back and forth about the axis ofrotation 214. More particularly, when the square wave is high forexample, the magnetic field set up by the magnetic circuits through thestator 218 and magnets 226 and 228 cause an end of the mirror to beattracted to the stator 218. At the same time, the magnetic fieldcreated by the magnetic circuits extending through the stator 220 andthe magnets 226 cause the opposite end of the mirror 212 to be repulsedby the stator 220. Thus, the mirror is caused to rotate about the axisof rotation 214 in one direction. When the square wave goes low, themagnetic field created by the stator 218 repulses the end of the springplate 216. At the same time, the stator 220 attracts the other end ofthe spring plate 216. Both forces cause the mirror 212 to rotate aboutthe axis 214 in the opposite direction.

In alternative embodiments, the scanning subsystem 14 instead includesacousto-optical deflectors, electro-optical deflectors, rotatingpolygons or galvanometers to perform the horizontal and vertical lightdeflection. In some embodiments, two of the same type of scanning deviceare used. In other embodiments different types of scanning devices areused for the horizontal scanner and the vertical scanner.

Eyepiece

Referring to FIGS. 2-4 the eyepiece 20 typically is a multi-element lensor lens system receiving the light beam(s) prior to entering the eye E.In alternative embodiments the eyepiece 20 is a single lens (see FIGS.5-7). The eyepiece 20 serves to relay the rays from the light beam(s)toward a viewer's eye. In particular the eyepiece 20 contributes to thelocation where an exit pupil of the scanning display 10 forms. Theeyepiece 20 defines an exit pupil at a known distance d from theeyepiece 20. Such location is the approximate expected location for aviewer's eye E.

In one embodiment the eyepiece 20 is an occluding element which does nottransmit light from outside the display device 10. In an alternativeembodiment, an eyepiece lens system 20 is transmissive to allow a viewerto view the real world in addition to the virtual image. In yet anotherembodiment, the eyepiece is variably transmissive to maintain contrastbetween the real world ambient lighting and the virtual image lighting.Referring to FIG. 10, a photosensor 300 detects an ambient light level.Responsive to the detected light level, a control circuit 302 varies abias voltage across a photochromatic material 304 to change thetransmissiveness of the eyepiece 20. Where the ambient light level isundesirably high, the photochromatic material 304 blocks a portion ofthe light from the external environment so that the virtual image ismore readily perceivable.

Optics Subsystem

Returning to FIGS. 2-7, the optics subsystem 14 receives the lightoutput from the light source, either directly or after passing throughthe scanning subsystem 16. In some embodiments the optical subsystemcollimates the light. In another embodiment the optics subsystemconverges the light. Left undisturbed the light converges to a focalpoint then diverges beyond such point. As the converging light isdeflected, however, the focal point is deflected. The pattern ofdeflection defines a pattern of focal points. Such pattern is referredto as an intermediate image plane.

According to an aspect of the invention, the optics subsystem 14includes an optical device for varying the curvature of light over time.Specifically the curvature pattern of the light entering the eye E forany given image element is controlled via the variable focus lens 22. Insome embodiments the lens 22 has its focus varied by controlling thethickness of the lens 22. In other embodiment the lens 22 has its focusvaried by varying the index of refraction of the lens 22.

The curvature of the light exiting lens 22 is controlled by changing theshape of the lens 22 or by changing the index of refraction of the lens22. A lens which changes its shape is shown in FIG. 11 and will bereferred to as an electro-mechanically variable focus lens (VFL) 320. Acentral portion 322 of the VFL 320 is constructed of a piezoelectricresonant crystalline quartz. In operation, a pair of transparentconductive electrodes 324 provide an electrical field that deforms thepiezoelectric material in a known manner. Such deformation changes thethickness of the central portion 322 along its optical axis toeffectively change the focus of the VFL 320.

Because the VFL 320 is a resonant device, its focal length variesperiodically in a very predictable pattern. By controlling the time whena light pulse enters the resonant lens, the effective focal position ofthe VFL 320 can be controlled.

In some applications, it may be undesirable to selectively delay pulsesof light according to the resonant frequency of the VFL 320. In suchcases, the VFL 320 is designed to be nonresonant at the frequencies ofinterest, yet fast enough to focus for each image element.

In another alternative embodiment, the variable focus lens is formedfrom a material that changes its index of refraction in response to anelectric field or other input. For example, the lens material may be anelectrooptic or acoustooptic material. In the preferred embodiment, thecentral portion 322 (see FIG. 10) is formed from lithium niobate, whichis both electrooptic and acoustooptic. The central portion 322 thusexhibits an index of refraction that depends upon an applied electricfield or acoustic energy. In operation, the electrodes 324 apply anelectric field to control the index of refraction of the lithium niobatecentral portion 322. In another embodiment a quartz lens includes atransparent indium tin oxide coating.

In another embodiment shown in FIG. 12, a lens 330 includes acompressible cylindrical center 332 having a gradient index ofrefraction as a function of its radius. A cylindrical piezoelectrictransducer 334 forms an outer shell that surrounds the cylindricalcenter 332. When an electric filed is applied to the transducer 334, thetransducer 334 compresses the center 332. This compression deforms thecenter 332, thereby changing the gradient of the index of refraction.The changed gradient index changes the focal length of the center 332.

In another embodiment shown in FIG. 13 the variable focus element is asemiconductor device 350 that has an index of refraction that dependsupon the free carrier concentration in a transmissive region 352.Applying either a forward or reverse voltage to the device 350 through apair of electrodes 354 produces either a current that increases thefree-carrier concentration or a reverse bias that depletes the freecarrier concentration. Since the index of refraction depends upon thefree carrier concentration, the applied voltage can control the index ofrefraction.

In still another embodiment shown in FIG. 14 a plurality of lenses360-362 are cascaded in series. One or more piezoelectric positioners364-366 move one or more of the respective lenses 360-362 along thelight path changing the focal distance of the light beam. By changingthe relative position of the lenses to each other the curvature of thelight varies.

One use of the variable focus lens 22 is to correct the curvature of anintermediate image plane for errors introduced by the scanning system 16or for aberrations introduced by other optical elements. For example, inthe embodiment of FIG. 13 a aberration map of the overall optical pathis stored in a look-up table in memory 370. The aberration map is a setof determined correction data representing the desired amount orvariation in the focal length of the variable focus element for eachpixel of an image. Control electronics 372 retrieve a value from thetable for each pixel and apply a corresponding voltage or other input toadjust the focal depth to correct for the aberration.

Light Source That Moves to Vary Light Wave Curvature

FIGS. 15 and 16 show embodiments of a scanning display 50/50′ in whichthe light source 13 includes one or more moving point sources 15. FIG.15 shows a display device 50 having an optics subsystem 14 and eyepiece20 that collimates the light. FIG. 16 shows a display device 50′ havingan optics subsystem 14 and eyepiece 20 that converges the light. In eachof the embodiments of FIGS. 15 and 16, the point sources 15 move alongan axis 54 normal to a plane of the optics subsystem 14. Thus, the pointsources 15 are moved either closer to or farther from the optics 14. Thechanging distance between the point source 15 and the optics 14 changesthe apparent distance of the point source 15 as viewed through the lens14. Moving the point source in one direction causes a virtual imageportion to appear farther away to the viewer. Moving the point source 15in the opposite direction causes the virtual image portion to appearcloser to the viewer. This is represented by the varying curvature ofthe light wavefronts 56 shown in FIGS. 15 and 16. By controlling thedistance of the point source 15 from the optics 14 the focus of an imageportion varies.

Responsive to a control signal, a position controller 60 determines thedistance from the point source 15 to the optics 14 for each pixel orgroup of pixels. In one embodiment, the controller 60 includes apiezoelectric actuator that moves the point sources 15. In anotherembodiment the controller 60 includes an electromagnetic drive circuitthat moves the point sources 15. The axis of motion of actuator or drivecircuit is aligned with the direction at which the point sources 15 emitlight, so that motion of the point sources 15 does not produce shiftingof the location of the respective pixel in the user's field of view.

FIG. 17 shows an embodiment for moving the apparent location of thepoint source 15. Light emitted from a light source 12 impinges on apartially reflective surface 122 that deflects the light toward a mirror124. The mirror 124 reflects the light back through the partiallyreflective surface 122, which transmits the light to the optics 14. Theangle at which the light impinges the optics 14 is determined by theorientation of the mirror 124. Such orientation is adjustable. In oneembodiment the mirror 124 is movable about a pivot line 126. In aninitial position the mirror 124 orientation is normal to the lightimpinging its surface. For a movement of the mirror 124 by an angle δzthe focal point of the light exiting the optics 14 varies by a distanceΔz and a height Δh. For a mirror 124 which receives the light at adistance w much greater than the arc distance δz, the distance Δz ismuch greater than the change in height Δh. Accordingly, the height Δhdifferential is not significant for many applications. Rotation of themirror 124 thus varies the focal distance for each image pixel withoutsignificantly affecting the apparent location of the pixel.

FIG. 18 shows a light source 13′ according to another embodiment of thisinvention. The light source includes a light emitter 15 that emits abeam of light. In one embodiment the light emitter 15 is a laser diode.In another embodiment, the light emitter 15 is a light emitting diodewith optics for making the output light coherent.

The light emitter 15 is carried by a support 64. In one embodiment thesupport 64 is formed of spring steel and is a cantilever type of spring.The cantilever spring has a spring constant determined by its length,width and thickness. Preferably, the support 64 is resonant with a highQ value such that once the support starts moving very little energy islost. As a result, very little energy is added during each period ofmovement to maintain a constant amplitude of motion of the support 64.For a high Q system the energy loss per cycle is less than 0.001%. Thesupport 64 is anchored at one end 65 and is free at an opposite end 67.Preferably, a position sensor monitors the position of the support 64and light emitter 15. In some embodiments a common mode rejectionpiezoelectric sensor 68 is used. In other embodiments a sensor 70responsive to changing inertia is used. An exemplary sensor 68 isdescribed in such U.S. Pat. No. 5,694,237 issued Dec. 2, 1997 entitled“Position Detection of Mechanical Resonant Scanner Mirror.”The lightsource 13′ also includes a base 76, a cap 78 and an electromagneticdrive circuit 60, formed by a permanent magnet 82 and an electromagneticcoil 84. The anchored end 65 of the support 64 is held to the permanentmagnet 82 by the cap 78. The permanent magnet 82 is mounted to the base76. The electromagnetic coil 84 receives the control signal causing amagnetic field to act upon the support 64. In another embodiment apiezoelectric actuator is used instead of an electromagnetic drivecircuit. The drive circuit 60 moves the support 64 and light emitter 15along an axis 88 way from or toward the optics 14 (of FIG. 15 or 16) tovary the focal distance of the light exiting the display.

In some embodiments the controller 60 moves the light emitter 15 togenerate a flat post-objective scan field. In effect the controllervaries the focal point of the emitted light to occur in a flatpost-objective image plane for each pixel component of an intermediaryimage plane 18 (see FIG. 19). FIG. 19 shows a point source 15 at threepositions over time, along with three corresponding focal points F1, F2and F3 along an intermediary image plane 18.

In another embodiment the curvature of the intermediary real image isvaried to match the curvature of an eyepiece 20′ as shown in FIG. 20. Asthe position of the light emitter 15 varies, the curvature of the imagelight 110 varies. As the light is scanned along the eyepiece 20′, thecurvature of the light is varied to match the curvature of the eyepiece20′ at the region where the light impinges the eyepiece 20′. FIG. 20shows a first curvature 112 for one position of the light emitter 15 anda second curvature 114 for another position of the light emitter 15.

Augmented Scanning Beam Display

FIG. 21 shows a preferred embodiment in which the scanning beam displayis an augmented display 150 which generates a virtual image upon abackground image. The background image may be an ambient environmentimage or a generated image. The virtual image is overlaid upon all or aportion of the background image. The virtual image may be formed ofvirtual two-dimensional or three-dimensional objects which are to beplaced with a perceived two-dimensional or three-dimensional backgroundimage environment. More specifically, virtual objects are displayed tobe located at an apparent distance within the field of view.

As previously described, the display device controls the curvature ofscanning light waves impinging on the eye to simulate image points ofdiffering depth. Images at far distances out to infinity have flat lightwaves impinging the eye. Images at near distances have convex-shapedlight waves impinging the eye. To simulate an object at a far distancethe light waves transmitted from the display to the eye are flat. Tosimulate closer objects, the light wave curvature increases. The eyeresponds to the changing curvature of the light waves by altering itsfocus. The curvature of the generated light waves relates to a desiredapparent focal distance between a virtual object and the eye.

The augmented scanning beam display 150 receives an image signal 152from an image source 154. The display 150 includes an image datainterface 11, one or more light sources 12, a lensing or opticssubsystem 14, a scanning subsystem 16, a beamsplitter 156, a concavemirror 158 and an eyepiece 20. Like parts performing the same or similarfunctions relative to the display 10 of FIG. 1 are given the same partnumbers. In one embodiment, the beamsplitter 156 and mirror 158 serve asthe eyepiece. In other embodiments another lens (not shown) is includedto serve as eyepiece 20.

The image source 154 which generates the image signal 152 is a computerdevice, video device or other digital or analog image data source. Theimage signal 152 is an RGB signal, NTSC signal, VGA signal, SVGA signal,or other formatted color or monochrome video or image data signal. Inresponse to the image signal 152, the image data interface 11 generatesan image content signal 160 for controlling the light source 12 and oneor more synchronization signals 162 for controlling the scanningsubsystem 16.

The light source 12 includes one or more point sources of light. In oneembodiment red, green, and blue light sources are included. In oneembodiment the light source 12 is directly modulated. That is, the lightsource 12 emits light with an intensity corresponding to the imagecontent signal 160. In another embodiment the light source 12 outputslight with a substantially constant intensity that is modulated by aseparate modulator in response to the signal 160. Light 164 is outputfrom the light source 12 along an optical path, being modulatedaccording to the image data within the image content signal 160. Suchmodulation defines image elements or image pixels. Preferably theemitted light 164 is spatially coherent.

The light 164 is output to the optics subsystem 14 and the scanningsubsystem 16. The scanning subsystem 16 includes a horizontal scannerand a vertical scanner. In one embodiment, the horizontal scannerincludes a mechanical resonator for deflecting passing light. Typicallythe light is deflected along a raster pattern, although in analternative embodiment another display format such as vector imaging canbe used.

The scanning subsystem 16 deflects the light along a raster patterntoward the eye E, or as in the embodiment illustrated, toward thebeamsplitter 156. The beamsplitter 156 passes both background light 166and virtual image light 168 to the viewer's eye E. The concave mirror158 focuses the light onto the eye E. The eye perceives the backgroundimage and an overlaid virtual image. The image pixels forming thevirtual image are scanned onto the viewer's eye. When the virtual imageis updated and rescanned periodically at a requisite frequency, theviewer perceives a continuous, virtual image.

The augmented display 150 also includes one or more light sensors 170,172 and a controller 174. Referring to FIGS. 21 and 22, light sensor 170detects the intensity of the background light 166. The controller 174receives the detected light intensity and generates a signal 176 whichin response adjusts the intensity of the virtual image light 168. In oneembodiment the virtual image light 168 intensity is adjusted bycontrolling the intensity of light 164 output by the light source 12.For example, controller 174 outputs a control signal 176 to the lightsource 12 to vary the light source 12 intensity.

Sensor 172 detects the distance of a background object or other focalviewing point of the background image light 166. Such sensor 172 is aconventional sensor of the kind used in cameras for determining objectdistance in connection with a camera's autofocus function. Thecontroller 174 with the sensor 172 generates a signal 178 forcontrolling the apparent distance of a virtual object to be overlaidupon the background object. In one embodiment the control signal 178 isinput to the variable focus lens 22 to adjust the curvature of the lightwaves forming the virtual image light 168. In an alternative embodiment,the control signal 178 moves the light source 12 to vary the curvatureof the light waves forming the virtual image light 168. In someembodiments, multiple sensors 172 are included for measuring backgrounddistance for many points within the background viewing field. Themeasuring points correspond to differing areas within the field of view.The measured distance for a given area is used to specify a distance fora virtual object to be overlaid upon the corresponding image area.Although, the term overlaid is used, the virtual object may be in partoverlaid and in part underlaid relative to a background object orbackground image area, as desired. Accordingly, a virtual image area isgenerated having an apparent distance which is correlated to a realworld image, and more particularly, to a real world image distance. Moregenerally, a virtual image area is generated having an apparent distancewhich is correlated to a background image, and more particularly, to abackground image distance.

For varying applications, in addition to controlling the content andpositioning of a virtual object, the object's shading, shadowing andother imaging effects can be controlled to achieve a desired realistic,surrealistic, or non-realistic effect. For example, in a gamingapplication virtual scenes may be superimposed upon a player's immediatebackground environment (e.g., the player's home, the woods, et cet.). Ina flight simulator, simulated terrain may be the source of thebackground image light, while simulated aircraft, targets or otherobjects may serves as the virtual objects. In such example, the terrainsimulator replaces or provides the inputs to the sensors 170, 172.

In some embodiments, the background area onto which an opaque virtualobject is overlaid is blanked. Commonly-assigned U.S. patent applicationSer. No. 09/009,759 of Charles D. Melville entitled, Augmented ImagingUsing A Silhouette To Improve Contrast, filed Jan. 20, 1998 isincorporated herein by reference and made a part hereof. Suchapplication describes the use of a silhouette display to blank out areasof background light to improve the contrast for a virtual image area.

Although preferred embodiments of the invention have been illustratedand described, various alternatives, modifications and equivalents maybe used. Therefore, the foregoing description should not be taken aslimiting the scope of the inventions which are defined by the appendedclaims.

What is claimed is:
 1. A scanning display apparatus, comprising: animage signal source operative to produce an image signal; a focalcontrol signal source generating a focal control signal; a light emittercoupled to the image signal source and responsive to the image signal toemit light; a lens which receives light from the light emitter and whichpasses exiting light, the exiting light having a focal distance; and acontroller responsive to the focal control signal for controllingdistance between the light emitter and the lens by moving the lensrelative to the light emitter without deforming the lens, wherein thefocal distance of the light exiting the lens varies with the distancebetween the light emitter and the lens.
 2. The apparatus of claim 1, inwhich the controller comprises an electromagnetic drive circuit.
 3. Theapparatus of claim 1, in which the controller comprises a piezoelectricactuator.
 4. The apparatus of claim 1, in which the light emitter is oneof a plurality of light emitters coupled to the image signal source andresponsive to the image signal to emit light toward the lens.
 5. Theapparatus of claim 1, serving as an augmented display, and furthercomprising a beamsplitter which receives the exiting light and whichfurther receives background light.
 6. The apparatus of claim 1, furthercomprising a light sensor which detects intensity of the backgroundlight and a controller which responds to the detected intensity tocontrol intensity of the emitted light.
 7. A scanning display apparatus,comprising: an image signal source operative to produce an image signal;a focal control signal source generating a focal control signal; a lightemitter coupled to the image signal source and responsive to the imagesignal to emit light; a lens which receives light from the light emitterand which passes exiting light, the exiting light having a focaldistance; a controller responsive to the focal control signal forcontrolling distance between the light emitter and the lens, wherein thefocal distance of the light exiting the lens varies with the distancebetween the light emitter and the lens; and a signal source responsiveto the received background light which varies the focal control signalto correlate the controlled distance to the background light.
 8. Theapparatus of claim 7, further comprising a distance sensor which detectsdistance of an object within a background field of view from which thebackground light is received, and wherein the signal source isresponsive to the received background light from the object and variesthe focal control signal to correlate the controlled distance to thedetected distance of the object.
 9. A scanning display apparatus,comprising: an image signal source operative to produce an image signal;a focal control signal source generating a focal control signal; a lightemitter coupled to the image signal source and responsive to the imagesignal to emit light; a mirror receiving the light from the lightemitter, the mirror movable about an axis in response to the focalcontrol signal to vary an angle at which the light is reflected from themirror; and a lens which receives light from the mirror and which passesexiting light, the exiting light having a focal distance, wherein theangle of the mirror determines the focal distance of light exiting thelens.
 10. The apparatus of claim 9, in which the light emitter is one ofa plurality of light emitters coupled to the image signal source andresponsive to the image signal to emit light toward the lens.
 11. Ascanning display apparatus, comprising: an image signal source operativeto produce an image signal; a focal control signal source generating afocal control signal; a light emitter coupled to the image signal sourceand responsive to the image signal to emit light; a lens which receiveslight from the light emitter and which passes exiting light, the exitinglight having a focal distance; and a controller responsive to the focalcontrol signal for controlling distance between the light emitter andthe lens by moving the light emitter relative to the lens, wherein thefocal distance of the light exiting the lens varies with the distancebetween the light emitter and the lens.
 12. The apparatus of claim 11,in which the controller comprises an electromagnet drive circuit. 13.The apparatus of claim 11, in which the controller comprises apiezoelectric actuator.
 14. The apparatus of claim 11, in which thelight emitter is one of a plurality of light emitters coupled to theimage signal source and responsive to the image signal to emit lighttoward the lens.
 15. The apparatus of claim 11, serving as an augmenteddisplay, and further comprising a beamsplitter which receives theexiting light and which further receives background light.
 16. Theapparatus of claim 11, further comprising a light sensor which detectsintensity of the background light and a controller which responds to thedetected intensity to control intensity of the emitted light.